Systems and methods for modeling energy consumption and creating demand response strategies using learning-based approaches

ABSTRACT

According to various implementations, a demand response (DR) strategy system is described that can effectively model the HVAC energy consumption of a house using a learning based approach that is based on actual energy usage data collected over a period of days. This modeled energy consumption may be used with day-ahead energy pricing and the weather forecast for the location of the house to develop a DR strategy that is more effective than prior DR strategies. In addition, a computational experiment system is described that generates DR strategies based on various energy consumption models and simulated energy usage data for the house and compares the cost effectiveness and energy usage of the generated DR strategies.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/944,669 filed Feb. 26, 2014, and entitled “Systems and Methodsfor Modeling Energy Consumption and Creating Demand Response StrategiesUsing Learning-Based Approaches,” the content of which is hereinincorporated by reference in its entirety.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under NSF #1059265project awarded by U.S. National Science Foundation. The government hascertain rights in the invention.

BACKGROUND

Electricity consumption in residential markets will undergo fundamentalchanges in the next decade due to the emergence of smart appliances andhome automation. A key requirement for the smart appliances within thesmart grid framework is the demand response (DR). The North AmericanElectric Reliability Corporation has defined demand response as changesin electricity usage by end-use customers from their normal consumptionpatterns in response to changes in the price of electricity(price-responsive DR), or to incentive payments designed to induce lowerelectricity use at time of high wholesale market prices or when systemreliability is jeopardized (curtailable DR).

Electric utility companies typically use hourly real-time price (RTP) orday-ahead price (DAP) structure in their dynamic pricing programs. InNorth America, Ameren Focused Energy, serving about 2.4 million electriccustomers in Illinois and Missouri, has very detailed RTP and DAPtariffs posted on their website since Jun. 1, 2008 for both day-aheadand real-time markets. The day-ahead market produces financially bindingschedules for the production and consumption of electricity one daybefore the operating day. The real-time market reconciles anydifferences between the amounts of energy scheduled day-ahead and thereal-time load, market participant re-offers, hourly self-schedules,self-curtailments and any changes in general, real-time systemconditions. Therefore, the DAP structure provides valuable informationfor price-sensitive loads while the RTP structure gives usefulinformation for curtail operations.

For a typical home in the United States, home appliances are responsiblefor an important part of the energy bills. These appliances may includehome heating, ventilation, and air conditioning system (HVAC), waterheaters, clothes washers and dryers, dishwashers, refrigerator andfreezers, electric stoves and/or ovens, coffee maker, home electricdrive vehicle charging system, and lights, for example. For each energyconsuming appliance, key factors affecting household energy consumptioninclude: 1) appliance load level, 2) when and how long an appliance isused, and 3) how much unwanted heat could be generated when using theappliance. For a flat electricity price structure, a customer would usean appliance whenever it is needed. However, for a dynamic electricityprice structure, customers are encouraged to optimize energy consumptionof their DR capable appliances.

Typically, the HVAC system is one of the more challenging appliances fora DR strategy. Traditionally, the thermostat of a HVAC unit is set at71° or 72° for a typical house in the United States. However, in adynamic price framework, the thermostat setting is regulated accordingto the real-time price information.

In modeling energy consumption of a residential house, the amount ofenergy consumed by the HVAC system is typically the most dominant part.The heat load that the HVAC system must overcome is mainly generated inthree ways: conduction, convection, and radiation. In most conventionalDR studies, the heat load of a residential house is computed based onsimplified approaches that typically only consider conduction. However,actual energy consumption of a residential house is much morecomplicated, which can be affected by geographical location, designarchitecture, window arrangements, insulation materials, occupants,weather, season, etc. and can change from one day to another.

A key component for a successful price-responsive DR program is a homeautomation system (HAS). Basically, a HAS receives information aboutweather forecast, dynamic electricity pricing, device operatingcharacteristics, usage requests, etc., and autonomously makes controldecisions and sends control actions to smart appliances.

However, a great challenge for the HAS to establish an optimalprice-responsive DR strategy is how to accurately model and estimate theenergy consumption of a residential house in variable weather conditionsand a dynamic pricing environment. As noted above, in existingtechnologies, many DR techniques are developed based on simplifiedenergy consumption models. For example, a strategy to minimize the costfor electricity consumption has been proposed in which the energyconsumption of a house is modeled based on simple conduction heattransfer equations. For example, a simplified equivalent and thermalparameter (ETP) modeling approach is used in GridLAB-D, a distributionsystem simulator, to estimate thermal loads of a residential house basedon first principles. Further, a quasi-steady-state approach has beenadopted to estimate hourly building electricity demand, in which thebuilding thermal model is built based on an equivalentresistance-capacitance network. The hourly energy consumption isdetermined through an optimization strategy under the constraints ofseveral predefined customer load levels which include maximum andminimum hourly demands, minimum daily consumption, and ramping up anddown limits.

Other approaches to DR strategies include a heuristic approach and an“optimal” DR approach. FIG. 1 illustrates a flow diagram of a heuristicDR strategy for operation of an HVAC system during the summer. Theheuristic DR strategy is a variable temperature setting approach. Duringthe summer time, for instance, the air conditioner should be operatedthe coolest possible near the lower boundary, T_(min) of the ASHRAEsummer comfort zone when the RTP is lower than a predefined value. Onthe other hand, the HVAC is operated the hottest possible near the upperboundary, T_(max), of the ASHRAE summer comfort zone. The HVAC isoperated between these two boundaries depending on the RTP tariff. InFIG. 1, P_(real) is the RTP for the current time frame i. P_(min) is aminimum price point, P_(max) is a maximum price point, and P₁-P_(n) areintermediate price points between P_(min) and P_(max).

Assuming there are n temperature settings between T_(max) and T_(min),then, the price and thermostat settings for summer time are calculatedby equations (1) and (2) below.

P _(i) =P _(max) −PR _(diff) tan h(k·i·PR _(diff))  (1)

T _(i) =T _(max)−(T _(max) −T _(min))/n·i  (2)

T _(i) =T _(max)+(T _(max) −T _(min))/n·i  (3)

The basic concept is that the thermostat setting is determined through acombined consideration of maximum and minimum real-time price and pricedistribution over a day. In equation (1) above, k is a constant obtainedfrom a price distribution study for different seasons, andPR_(diff)=P_(max)−P_(min), where P_(max) and P_(min) correspond tomaximum and minimum electricity prices of a day, respectively. Forwinter time, a modification is necessary with T_(max) and T_(min)corresponding to P_(min) and P_(max), respectively, and the price andthermostat settings are calculated by equations (1) and (3) above.Similarly, T₁, T₂, and T_(n) correspond to P₁, P₂, and P_(n),respectively.

The DR strategy obtained according to FIG. 1 may be good some days butbad other days. Another problem with the heuristic DR strategy is thatthe algorithm is unable to take the advantage of low price periods topre-cool down a house significantly.

The optimal DR approach attempts to determine the optimal thermostatsetting strategy for a given dynamic price of a day, and thus is moreefficient than the heuristic DR approach. To develop an optimal DRstrategy, the following nonlinear programming formulation is used:

$\begin{matrix}{{{{Minimize}:\mspace{14mu} C} = {\sum\limits_{i}\; {p_{i} \cdot Q_{i}}}}{{{{Subject}\mspace{14mu} {to}\text{:}\mspace{14mu} 0} \leq Q_{i} \leq Q_{\max}},{T_{\min} \leq T_{i}^{I} \leq T_{\max}}}} & (4)\end{matrix}$

where T_(min)=T_(ideal)−d, T_(max)=T^(deal)+d, d is the acceptabletemperature deviation, i represents a time slot in one hour, T_(i) ^(I)is the room temperature in hour i, C is the electricity cost during aday, p_(i) stands for the electricity price in hour i, Q_(i) signifiesthe energy consumed by the HVAC unit in hour i, and Q_(max) denotes themaximum energy that can be consumed by the HVAC unit. Traditionally, theenergy consumed by the HVAC Q_(i) is modeled as a function of roomtemperature T_(i) ^(I) and outdoor temperature T_(i) ^(O). Equation (5)below shows a simplified thermal model of a residential house:

T _(i+1) ^(I) =ε·T _(i) ^(I)+(1−ε)(T _(i) ^(O) −η·Q _(i) /A)  (5)

where η is the efficiency of the HVAC unit, Σ is the system inertia, andA is the thermal conductivity. However, in reality, the relationshipbetween indoor and outdoor temperatures and energy consumed by the HVACunit is much more complicated than Equation (5) so that actual energyconsumption of the HVAC unit could deviate greatly from resultsgenerated by using Equation (5), which affects the DR efficiency.Therefore, an intelligent mechanism that can identify and update anenergy consumption model daily for a residential house is needed fordeveloping an optimal DR strategy.

BRIEF SUMMARY

According to various implementations, a DR strategy development systemis described that can effectively model the energy consumption ofregulatable energy consumption units in a house using a learning basedapproach that is based on actual energy usage data collected over aperiod of days. This modeled energy consumption may be used withday-ahead energy pricing and the weather forecast for the location ofthe house to develop a DR strategy that is more effective than prior DRstrategies.

According to certain implementations, a system for creating a demandresponse strategy for a building is provided. The system includes acomputing device that includes a memory and a processor. The memory isconfigured for storing actual usage data associated with an energyconsuming unit in the building for one or more time windows on each ofdays i through j, wherein day i is the first day for which data isstored and j is the most recent day for which data is stored. Theprocessor is configured for: (1) receiving the actual usage data fromthe memory; (2) executing at least one computer-based learning system tomodel energy consumption for day j+1 based on at least the actual usagedata for the energy consumption unit, and (3) generating a demandresponse strategy for the energy consuming unit for day j+1 based on themodeled energy consumption and next-day energy pricing for each timewindow for day j+1. The computer-based learning system may include aneural network system or a regression based system, for example.

In one implementation, the energy consumption unit is an HVAC system,and the actual usage data includes an outdoor temperature for each timewindow on days i through j, an indoor temperature for each time windowon days i through j, and a thermostat setting for each time window ondays i through j. In this implementation, the processor is furtherconfigured for using one or more of the computer based learning systemsto model energy consumption of the HVAC system for day j+1 based on theweather forecast for day j+1 and the actual usage data for days ithrough j. In addition, generating the demand response strategy includesgenerating thermostat settings for one or more time windows on day j+1such that energy costs for day j+1 are minimized.

According to another implementation, an energy consumption managementsystem is provided. The energy consumption management system includesone or more local receivers and a central computer computing system. Theone or more local receives are disposed adjacent a respective one of oneor more energy consuming units in a building, and each local receiverincludes a processor configured for receiving usage instructions for theadjacent energy consuming unit and causing the usage instructions to beexecuted for the energy consuming unit. The central computing systemincludes a memory and a processor. The memory is configured for storingactual usage data associated with at least one energy consuming unit inthe building for one or more time windows on each of days i through j,wherein day i is the first day for which data is stored and day j is themost recent day for which data is stored. The processor is configuredfor: (1) receiving the actual usage data from the memory; (2) executingat least one computer-based learning system to model energy consumptionfor day j+1 based on at least the actual energy usage data for theenergy consumption unit; (3) generating a demand response strategy forthe energy consuming unit for day j+1 based on the modeled energyconsumption and next-day energy pricing for each time window for dayj+1; and (4) communicating the demand response strategy of the energyconsuming unit to the local receiver associated with the energyconsuming unit. The demand response strategy comprises the usageinstructions. The computer-based learning system may include a neuralnetwork system or a regression based system, for example.

According to yet another implementation, a computational experimentsystem for comparing demand response strategies is provided. The systemincludes a computing device, and the computing device includes a memoryand a processor. The memory is configured for storing simulated usagedata associated with an energy consuming unit in a building for one ormore time windows on each of days i through j, wherein day i is thefirst day for which data is stored and j is the most recent day forwhich data is stored. The processor is configured for: (1) receiving thesimulated usage data from the memory; (2) executing at least onecomputer-based learning system to model energy consumption for day j+1based on the simulated usage data; (3) generating a first demandresponse strategy for day j+1 based on the modeled energy consumptionand next day energy price data; (4) generating a second demand responsefor day j+1; and (5) generating a display indicating energy costsassociated with the first and second demand response strategies for dayj+1. The computer-based learning system may include a neural networksystem, a regression based system, a non linear programming approach,and/or a particle swarm optimization approach, for example.

The second demand response strategy may be generated using a heuristicapproach based on real time pricing data, according to someimplementations. In other implementations, the energy consuming unit isan HVAC unit having an efficiency η, system inertia ε, and thermalconductivity A, and the second demand response strategy is based on thefollowing simplified thermal model approach:

T _(i+1) ^(I) =ε·T _(i) ^(I)+(1−ε)(T _(i) ^(O) −η·Q _(i) /A).

In other implementations, the second demand response strategy may begenerated using a nonlinear programming approach or a particle swarmoptimization approach.

The processor may also be configured for: (1) receiving an energy pricefor each time window on day j+1, (2) generating a demand responsestrategy for the energy consuming unit for day j+1 based on the energyprice for each time window on day j+1, (3) calculating a respective costfor operating the energy consuming unit on day j+1 based on the firstand second demand response strategies and the energy pricing, and (3)displaying a projected cost for energy usage for the energy consumptionunit associated with each respective schedule associated with eachenergy consumption model.

BRIEF DESCRIPTION OF THE DRAWINGS

The components in the drawings are not necessarily to scale relative toeach other and like reference numerals designate corresponding partsthroughout the several views:

FIG. 1 illustrates a flow chart of a heuristic DR strategy for operationof an HVAC system during the summer.

FIG. 2 illustrates a schematic diagram of a DR strategy system for anHVAC unit according to one implementation.

FIG. 3 illustrates a schematic network diagram of the DR strategy systemof FIG. 2.

FIG. 4 illustrates a schematic diagram of an exemplary central serverconfigured to implement at least a portion of the DR strategy systemdescribed in FIGS. 2 and 3, according to one implementation.

FIGS. 5A-5B illustrate a flow diagram of steps taken by various portionsof the DR strategy system according to one implementation.

FIG. 6 illustrates a schematic diagram of the flow of data through aneural network according to one implementation.

FIG. 7 illustrates a schematic diagram of an exemplary central serverconfigured to implement at least a portion of a computational experimentsystem according to one implementation.

FIGS. 8A-8B illustrate a flow diagram of steps taken by various portionsof the computational experiment system according to one implementation.

FIG. 9 illustrates a schematic diagram of a computational experimentsystem according to one implementation.

FIGS. 10-15 illustrate charts comparing simulated and modeled energyusage and energy pricing using various approaches according to variousimplementations.

FIG. 16 is a schematic diagram illustrating the flow of energy andinformation among various energy consuming and generating units and ahome energy management system according to one implementation.

FIG. 17 is a schematic diagram illustrating several components of asolar energy system for a building, according to one implementation.

FIG. 18 is a schematic diagram illustrating several components of thehome energy management system according to one implementation.

FIG. 19 illustrates a room inside of a building having variouselectrical components and sensors, according to one implementation.

DETAILED DESCRIPTION

According to various implementations, a DR strategy system is describedthat can effectively model the energy consumption and generation of ahouse using a learning based approach that is based at least in part onactual energy usage data collected over a period of days. This modeledenergy consumption may be used with day-ahead energy pricing and aweather forecast for the location of the house to develop a DR strategythat is more effective than prior DR strategies. One exemplary goal ofthe DR strategy system is to minimize net energy cost to the consumer.This goal may be met by reducing the amount of energy consumed duringpeak tariff time periods and identifying optimal time periods forstoring, using, or selling energy generated from renewable energysources associated with the house.

In particular, interactive learning mechanisms are disclosed that areconfigured to learn and model energy consumption of a residential houseusing substantially real-time measured or simulated energy consumptiondata. Due to the learning, the model can accurately capture the thermalbehavior of a house for different seasons, users, and weatherconditions. For example, the thermal storage characteristics of thehouse may be captured by the learning mechanisms. The model may then beused to develop a cost effective DR strategy for one or more energyconsuming units in the house, such as the HVAC system, other appliances,or other energy consuming devices in the house, and any energygenerating units associated with the house.

To more accurately model the energy consumption for a household andidentify an optimal DR strategy for the energy units in the home, energyusage and generation from regulatable loads, fixed deferrable loads, andfuture fixed non-deferrable loads and sources may be optimizedseparately and then combined to achieve a final, robust optimalsolution. Regulatable loads includes energy loads for which energy usagemay be regulated but not delayed. Examples of regulatable loads includethe HVAC system, hot water heater(s), and battery(s) storing electricitygenerated from renewable energy sources associated with the house (e.g.,solar photovoltaic panels, such as roof-top solar photovoltaic panels,and wind turbines). Fixed deferrable loads includes energy loads forwhich energy usage may be deferred but not regulated. Examples of fixeddeferrable loads include the clothes washer and dryer, dishwasher, andvehicle charging station. Future fixed non-deferrable loads includeenergy loads for which energy usage or generation cannot be regulated ordeferred. Examples of future fixed non-deferrable loads include lights,microwave, home office equipment, refrigerator, oven/stove, and energygenerated from any renewable energy sources associated with the house.

Energy units having regulatable loads and fixed deferrable loads aresubject to management by the HAS using a DR policy generated the daybefore. The DR policy may be adjusted in the current day based on realtime pricing (RTP) data, if available. For example, the DR policy for aregulatable load may be solved using linear or nonlinear programmingmethods, and DR policy for fixed deferrable loads may be solved using aninteger programming technique (e.g., binary integer programming).

However, energy usage by energy units having future fixed non-deferrableloads may not be deferred or regulated. Instead, energy usage by atleast a portion of these future fixed non-deferrable load energy unitsmay be optimized using sensor technology as part of the DR policy forminimizing energy usage. For example, a sensor may be associated withone or more energy consuming units having a future fixed non-deferrableload that turns off the unit when it is not needed or after apre-defined time period or switches the energy consuming unit into anenergy saving or sleep mode when the HEMS 10 senses that no people arein the room or senses that the appliance is not in use. Theimplementation shown in FIG. 19 includes a motion sensor, a door controlsensor, and a security sensor, for example.

The amount of energy generated by a photovoltaic (PV) system or otherrenewable resource (e.g., wind turbines) may be communicated to the HAS14, and the HAS 14 may determine whether to use, store, or sell theenergy generated based on the DR policy for the regulatable anddeferrable energy loads in the home, the RTP tariff for selling energyto the electrical grid, and the remaining capacity of the battery usedto store the energy for later use. In some implementations, the HAS 14may treat the energy generated by the renewable resource as a futurefixed generation source with a HAS generated dynamic pricing and theenergy stored by the battery 7 as a regulatable load that is negativewhen discharging and positive when charging with a HAS generated dynamicpricing.

The HAS 14 may also include strategies to learn energy generationproperties of renewable energy source systems, such as PVs and windturbines, energy storage characteristics of batteries, and dynamic pricetariffs for the home energy generation system and battery storage systemso that the energy generation system and energy storage can be operatedand coordinated with usage of other appliances for the maximum benefitof the home owner and to achieve the goal of having a zero energy homecapability as much as possible.

For example, various implementations provide an optimal DR strategy fora household heat pump, or HVAC unit, that is based on day-ahead weatherprediction, day-ahead electricity pricing, and a learned energyconsumption model of the house. The DR strategy for the HVAC unit forthe next day includes the thermostat settings for various time periodsfor the next day. The DR strategy for other appliances or energyconsuming units having a fixed deferrable load may include a RTP tariffrange for which operation of the unit is acceptable, a certain timeperiod during which the appliance or energy consuming unit may beoperated, and/or an acceptable operation cycle.

FIGS. 2, 3, and 16-18 illustrate various components of a home energymanagement system (HEMS) 10 according to various implementations. Inparticular, the system 10 includes a wireless home-area network 12 withan intelligent HAS 14 and one or more distributed wireless receivers 16a, 16 b, 16 c associated with each energy consuming unit 18 a, 18 b, 18c, such as an appliance, electric drive vehicle (EV) charging system,one or more lights, or home office equipment. The HEMS 10 may alsoinclude one or more wireless sensors 19 associated with energy unitshaving future fixed non-deferrable loads. The HEMS 10 manages energyusage for the energy consuming and generating units in the house.

The HAS 14 receives information from each distributed, local receiver 16a-16 c about energy usage of the energy consuming unit 18 a-18 c,respectively, over the home area network 12 and stores the data in amemory, such as queues 26 or other data structures, on a first in, firstout (FIFO) basis. The queues 26 are configured to hold data for days 1through n. In one implementation, n may be eight days. The data iscollected by the HAS 14 in substantially real-time, such as every hour,minute, second, or other appropriate time period. The HAS 14 may alsoreceive inside and outside temperatures 25 and thermostat settings forvarious time periods during the current day. The HAS 14 also receivespredicted weather 20 and electricity price information 22 for the nextday via an external network 24, such as the Internet.

To implement the DR policy for fixed deferrable loads, the HAS 14 mayinclude a control block and a next-day DR block. The control block forthe fixed deferrable loads provides control commands to local receiversat the present day while a local receiver defers the operation of anenergy consuming unit, based on the control commands received from theHAS 14. The next-day DR block for fixed deferrable loads may include anoptimization module that receives DAP tariff information from theutility company over a network, such as the Internet or private network,or a smart meter attached to the house as well as day-ahead dynamicprice tariffs determined by the HAS 14 for home solar PV system andbattery storage system and executes an optimization routine to identifythe best time slot to operate a deferrable unit for the next day. Incertain implementations, an integer programming strategy is used toidentify the most cost effective time slot(s) to operate the fixeddeferrable load units. Electric utility DAP tariff information mayinclude other non-RTP tariffs, such as time of use (ToU) tariffs andcritical peaking price (CPP) tariffs.

The HAS 14 may include a control block and a next-day DR block forregulatable loads. The control block for regulatable loads providescontrol commands to local receivers at the present day based on a DRpolicy generated one day ahead while a local receiver regulates theoperation of a regulatable energy unit, depending on the controlcommands received from the control block. The next-day DR block forregulatable loads receives information from a local receiver aboutenergy usage of an appliance at the present day as well as predictedweather and electricity price information and dynamic price tariffsdetermined by the HAS for the home solar PV system and battery storagesystem one day ahead and determines a DR policy for the next day. The DRpolicy determined by the next-day DR block for regulatable loadsimplements the demand response of those units for the maximum benefit ofthe residential consumer at any weather condition. The ability toidentify the energy consumption models of those units throughdata-driven learning mechanisms, such as those described above and shownin FIG. 2, is an important step for the next-day DR block, according tocertain implementations. For example, an intelligent learning module ofthe HAS 14 models energy consumption for one or more regulatable loadsbased on the data stored in the queues 26.

The next-day DR block for regulatable loads may include an optimizationmodule that creates a DR strategy for the next day that is based on themodeled next-day energy consumption and day-ahead energy pricing. Forexample, the DR strategies for the next day for regulatable units, suchas the HVAC unit and the hot water heater, include a plurality ofoperational settings (e.g., thermostat settings for the HVAC unit and/orhot water heater) corresponding to various time periods for the nextday. The next-day DR block for fixed deferrable loads may also includean optimization module that creates a DR strategy using the DAP or RTPdata and time windows and price ranges that are predefined by thecustomer as being acceptable for operation of one or more of the otherenergy consuming units. The queues 26 may also store data from theseenergy consuming units. The optimization modules may be furtherconfigured for adjusting the DR strategy for the current day usingreal-time energy pricing, if available.

Control commands based on the DR strategy are communicated by the HAS 14to local receivers 16 a-16 c via the home area network 12. The localreceivers 16 a-16 c may discontinue the operation of their associatedenergy consuming unit 18 a-18 c to a later time period having a cheaperenergy price or regulate the operation of the associated energyconsuming units 18 a-18 c, depending on the control commands from theHAS 14. FIG. 18 illustrates an exemplary implementation in which thelocal receivers 16 a-16 c include ZIGBEE nodes and smart switches.However, in other implementations, other types of receivers may be used.

The substantially real-time DR system 10 is adaptive to implement theoptimal demand response strategy for the maximum benefit of aresidential consumer insubstantially any weather conditions. A keycomponent of various implementations of the system is the ability toidentify a substantially real-time HVAC energy consumption model that isadaptive to weather and seasonal conditions through learning-basedcomputing approaches.

The HEMS 10 may also be configured for reporting usage conditions to auser, allow the user to activate or deactivate DR appliances, adjusttemperatures (or ranges thereof) for regulatable appliances, accept timeslot(s) for deferrable load appliances according to an incentive pricetariff from a utility company. The HEMS 10 may communicate with the uservia a wired interface or a wireless interface. For example, the HEMS 10may include a display screen through which the user receives informationfrom the HEMS 10 and an input device through which the user providesinstructions to the HEMS 10. In other implementations, the HEMS 10 maycommunicate with the user via a smart phone application or otherinterface that is wirelessly in communication with the HEMS 10.

Exemplary Learning-Based DR Strategy System

FIG. 4 illustrates a schematic diagram of a central server 500, orsimilar network entity, configured to implement a computer system,according to one implementation. The server 500 executes variousfunctions of the HEMS 10 described above in relation to FIGS. 2 and 3and below in relation to FIGS. 5A and 5B. For example, the server 500may be the HAS 14 described above, or a part thereof. As used herein,the designation “central” merely serves to describe the commonfunctionality the server provides for multiple clients or othercomputing devices and does not require or infer any centralizedpositioning of the server relative to other computing devices. As may beunderstood from FIG. 4, the central server 500 may include a processor510 that communicates with other elements within the central server 500via a system interface or bus 545. Also included in the central server500 may be a display device/input device 520 for receiving anddisplaying data. This display device/input device 520 may be, forexample, a keyboard or pointing device that is used in combination witha monitor. The central server 500 may further include memory 505, whichmay include both read only memory (ROM) 535 and random access memory(RAM) 530. The server's ROM 535 may be used to store a basicinput/output system 540 (BIOS), containing the basic routines that helpto transfer information across the one or more networks.

In addition, the central server 500 may include at least one storagedevice 515, such as a hard disk drive, a floppy disk drive, a CD Romdrive, or optical disk drive, for storing information on variouscomputer-readable media, such as a hard disk, a removable magnetic disk,or a CD-ROM disk. As will be appreciated by one of ordinary skill in theart, each of these storage devices 515 may be connected to the systembus 545 by an appropriate interface. The storage devices 515 and theirassociated computer-readable media may provide nonvolatile storage for acentral server. It is important to note that the computer-readable mediadescribed above could be replaced by any other type of computer-readablemedia known in the art. Such media include, for example, magneticcassettes, flash memory cards and digital video disks.

A number of program modules may be stored by the various storage devicesand within RAM 530. Such program modules may include an operating system510 and a plurality of one or more (N) modules 560. The modules 560 maycontrol certain aspects of the operation of the central server 500, withthe assistance of the processor 510 and the operating system 550. Forexample, the modules may perform the functions described above inrelation to FIGS. 2 and 3 and below in relation to FIGS. 5A-6 andillustrated by the figures and other materials disclosed herein, such asexecuting various functions of the DR strategy system 10. According tovarious implementations, one or more of the modules may be executed by adigital computing system or portion thereof, such as a micro computer,digital signal processor chip, field programmable gate array (FPGA)system, PC, or other suitable computing device.

In one exemplary implementation, the server 500 includes the followingmodules: (1) a communication module for receiving data relevant tomodeling energy consumption and energy pricing and communicating the DRstrategy to the local receivers associated with the energy consumingunits; (2) a learning module for modeling energy consumption based onrelevant data using a computer implemented learning-based approach; (3)an optimization module for creating a next day DR strategy forregulatable loads based on DAP data and the modeled energy consumptionfrom the learning module and for adjusting the DR strategy for thecurrent day based on RTP data for the current day; (4) an optimizationmodule for creating a next day DR strategy for fixed deferrable loadsusing the DAP or RTP data and time windows; and (5) a data managementmodule for storing in queues in a memory actual energy consumption data,temperature data (indoor and outdoor), and thermostat settings for eachtime window for 1 through n days.

Learning Module

As noted above, the learning module models energy consumption based onrelevant data using a computer-implemented learning-based approach. Thelearning-based approaches may include a neural network approach or aregression-based approach, for example. These exemplary approaches aredescribed below beginning with the description of FIG. 6. In addition,other suitable learning mechanisms may be used. And, these learningmechanisms may be used for modeling energy generation characteristics ofa renewable energy source, such as a PV system or a wind turbine system,and/or energy storage characteristics of a battery for storing energygenerated by the renewable energy source.

Neural Network Approach

In a neural-network-learning-based approach, the model of the energyconsumed by the HVAC Q_(i) is obtained through a neural-network-basedlearning mechanism. Unlike the fixed model shown by Equation 5, the HVACenergy consumption model learned by the neural network is updated daily.Hence, it can more accurately capture the thermal behavior of a house atdifferent seasons, users, weather conditions, etc. The neural network istrained by using a backpropagation algorithm, which includes multipleiterations until a stop criterion is reached. Thus, it is morecomputational expensive.

One neural network learning approach is a multilayer perceptron (MLP). AMLP is an artificial neural network structure and may be useful inmodeling HVAC energy consumption, for example. As shown in FIG. 6, anMLP includes a set of source nodes that make up the input layer, one ormore layers of computation nodes, and an output layer. The input signalpropagates through the network in a forward direction, on alayer-by-layer basis. The network exhibits a high degree ofconnectivity, determined by the weights of the network. Experientialknowledge for the network is acquired by the network through a learningprocess and stored in the network weights after it is trained.

For the learning purpose of HVAC energy consumption model, the MLP 30shown in FIG. 6 includes three input nodes 31 a, 31 b, 31 c, onecomputation layer 33 that includes eight nodes, and one output node 35.The three inputs nodes 31 a, 31 b, 31 c of the network 30 receive thefollowing data: 1) outside temperature T_(i) ^(O) in hour i (node 31 a),2) T_(i) ^(I) room temperature in hour i (node 31 b), and 3) T_(i+1)^(I) room temperature in hour i+1 (also represents the thermostatsetting temperature in hour i) (node 31 c). The network output is Q_(i),signifying the energy consumed by the HVAC unit in hour i (FIG. 6). Itis possible to add other inputs to the network to enhance the learning.

As shown in FIG. 6, the network learning is based on the data saved infour queues and is updated daily. The data saved in the four queues arecorresponding to outside temperature (T_(i) ^(O)), room temperature(T_(i) ^(I)), thermostat setting temperature (T_(i+1) ^(I)), and HVACenergy consumption (Q_(i)).

A MLP network can be used for a function approximation problem, in whichthe inputs to the network are equivalent to the predictor variables asshown in the regression model of Equation 8 below and the output of thenetwork is equivalent to the predicted value. For a given problem, thereis a cost function ε_(T), which is similar to the error sum of squaresof Equation 7 (shown below) for the regression model, as the measure oftraining set learning performance. The objective of the learning processis to adjust the weights of the network so as to minimize ε_(T). Ahighly popular training algorithm known as the backpropagation algorithmis generally used to adjust the network weights until a stop criterionis reached.

After the network is trained, the neural network would provide a modelthat describes the relation of the HVAC energy consumption (Q_(i)) withthe outside temperature (T_(i) ^(O)), room temperature (T_(i) ^(I)), andthermostat setting temperature (T_(i+1) ^(I)) at a time frame i. Thismodel is updated daily and then used by the optimization module todetermine an optimal next-day DR strategy to operate the HVAC for thenext 24 hours given the predicted outside temperature)(T_(i) ^(O) andelectricity price.

Regression Based Approach

In a regression-learning-based approach, the model of the energyconsumed by the HVAC Q_(i) is obtained through a regression-basedlearning mechanism. Compared to the fixed model in Equation 5, theregression model can also provide more accurate estimation of HVACenergy consumption that is close to the actual results, because themodel is updated daily through the regression-based learning mechanism.With regard to the neural-network-based model, the regression-basedlearning is much faster because the parameters of the regression modelcan be solved directly.

Regression models quantitatively describe the variability among theobservations by partitioning an observation into two parts. The firstpart of this decomposition is the predicted portion having thecharacteristic that can be ascribed to all the observations consideredas a group. The remaining portion, called the residual, is thedifference between the observed value and the predicted value and has tobe ascribed to unknown sources. This can be expressed as

y _(i)=ƒ(x _(i),β)=ε_(i) i=1, . . . ,n  (6)

where n is the number of the observations, y_(i) is the ith observation,x_(i)=(x₁, x₂, . . . , x_(k)) is the predictor variable vector relatedto observation y_(i), β=(β₀, β₁, . . . , β_(p)) is the parameter vector,and ε_(i) is the error associated with the ith observation.

The function ƒ(•) is assumed to be smooth and estimated by fitting apolynomial or other types of functions. Fitting refers to calculatingvalues of the parameters from a set of data. Similar to the neuralnetwork learning, the estimate {circumflex over (β)}, a least squaresestimate of β, tries to minimize the error sum of squares shown by.

$\begin{matrix}{{{S\left( \hat{\beta} \right)} = {\underset{i = 1}{å^{n}}\left( {y_{i} - {f\left( {x_{i},\hat{\beta}} \right)}} \right)}^{2}},} & (7)\end{matrix}$

Regression approach is very effective if one knows the general format ofa function that the observations would follow. Based on theoreticalstudies about building heat loads, a 3^(rd) order polynomial linearregression function is used to learn HVAC energy consumption model asdescribed mathematically by

Q _(j) =q(T _(j+1) ^(I) ,T _(j) ^(I) ,T _(j) ^(O),β) j=1, . . . ,n  (8)

where q(•) is a 3rd order polynomial function of (T_(j+1) ^(I),T_(j)^(I),T_(j) ^(O)), n is the number of the observations saved in the eachof the queues as shown in FIG. 2, Q_(j) is the jth HVAC energyconsumption observation, (T_(j+1) ^(I),T_(j) ^(I),T_(j) ^(O)) is thepredictor variable vector consisting of thermostat settings and indoorand outdoor temperatures related to observation Q_(j), and β=(β₁, β₁, .. . , β_(p)) is the parameter vector, wherein p represents the totalnumber of coefficients of the polynomial function. It is possible to addother inputs to the predictor variable vector to enhance the learning.Similar to the neural network approach, the regression model is updateddaily when new data is saved in the queues each day. Compared to thesimplified model Equation 5, the regression model can also provide moreaccurate estimation of HVAC energy consumption that is close to theactual results.

Similar to the neural network, the regression approach would provide amodel that describes the relation of the HVAC energy consumption (Q_(i))with the outside temperature (T_(i) ^(O)), room temperature (T_(i)^(I)), and thermostat setting temperature (T_(i+1) ^(I)) at a time framei. This model is updated daily and is then used by the optimizationmodule to determine an optimal next-day DR strategy to operate the HVACfor the next 24 hours given the predicted outside temperature)(T_(i)^(O)) and electricity price.

Model Based Approach

As discussed above, the energy consumption model of the HVAC can beobtained by using a learning based approach, such as the neural networkapproach or regression based approach described above. In addition, thelearning module may also model energy consumption by using a model-basedapproach. In the model based approach, the energy consumed by the HVACQ_(i) is modeled as a function of room temperature T_(i) ^(I), roomtemperature T_(i+1) ^(I) in hour i+1 (also represents the thermostatsetting temperature in hour i), and outdoor temperature T_(i) ^(O).Equation 5 above shows an example of a simplified thermal model of aresidential house. However, in reality, the relationship between indoorand outdoor temperatures and energy consumed by a HVAC unit is much morecomplicated than Equation 5 so that actual energy consumption of a HVACunit could deviate greatly from results generated by using Equation 5,which would affect the DR efficiency.

Optimization Modules

The optimization modules generate DR strategies for one or more energyunits having regulatable loads and/or fixed deferrable loads. Inparticular, the optimization module for creating DR strategies forregulatable loads may solve the optimization problem of Equation 4 tocreate a DR strategy for each regulatable energy unit to minimize thecosts of energy consumption for a particular day. For example, theoptimization module may use one of the following optimization approachesto solve the optimization problem: (1) a nonlinear or linear programmingapproach and 2) particle swarm technique. These are discussed in moredetail below in relation to FIGS. 5A and 5B and Algorithms 1 and 2.

The optimization module for creating DR strategies for fixed deferrableloads may generate a DR strategy for each fixed deferrable load that maybe based on real time energy pricing, acceptable price ranges, and/ortime frames during which these energy consuming units may be operated.The energy usage of fixed deferrable loads is considered to have a fixedenergy consumption pattern. The optimization module may use a binaryinteger programming technique to solve the optimization problem forfixed deferrable loads.

If assuming there are one HVAC load, one deferrable load, and one futurefixed load, the robust integrative optimization can be describedmathematically by

$\begin{matrix}{{{Minimize}\text{:}}{C = {\sum\limits_{i}\; {\left( {p_{i} - p_{i}^{PV} - p_{i}^{batt}} \right) \cdot \left( {Q_{i}^{HVAC} + Q_{i}^{deferrable} + Q_{i}^{fixed}} \right)}}}{{i = 1},\ldots \mspace{14mu},24}{{{{Subject}\mspace{14mu} {to}\text{:}\mspace{14mu} 0} \leq Q_{i}^{HVAC} \leq Q^{\max}},{T_{\min} \leq T_{i + 1}^{I} \leq T_{\max}},{Q_{i}^{HVAC} = {{{q\left( {T_{i + 1}^{I},T_{i}^{I},T_{i}^{O},\overset{\rightarrow}{w}} \right)}\mspace{14mu} i} = 1}},\ldots \mspace{14mu},24}} & (9)\end{matrix}$

in which p_(i) ^(PV) and p_(i) ^(batt) are day-ahead dynamic discountprice tariffs associated with PV and battery. The purpose of thediscount price tariffs is to generate a virtual low price signal to homeappliances so as to shift DR appliances to those “low price” timeframes, such as during a high PV generation time period during day timeor during a battery discharging time period after sunset. Similarly,computer-based learning systems may be configured for formulatingdynamic discount price tariffs associated with PV power production andbattery storage capability at different seasons, weather, and houseconditions. Then, the next step is to solve the overall optimizationproblem to determine an optimal DR policy to operate the homeappliances. Furthermore, the system may also generate a control andmanagement strategy for the battery for the next day based at least on acharge of the battery for the current day and an expected, or modeled,difference between next-day energy generation from at least onerenewable energy generation source and energy consumption from at leastone energy consuming unit. The control and management strategy mayinclude one or more time windows during which the battery is chargedwhen energy generation is higher than energy demand or when energy fromthe battery is discharged for consumption by one or more energyconsuming units when energy generation is not occurring or is occurringat a level below a predetermined acceptable threshold.

To solve the integrative optimization problem, many previous studiesfocused on developing methods to predict energy consumption patterns ofall home appliances, including fixed deferrable and fixed non-deferrableloads, to manage the loads in the DR framework. However, due to variousfactors, the forecast data are far from accurate. Thus, most of thesesolutions used stochastic optimization techniques to solve theoptimization problem. However, inventors discovered that knowing energyconsumption patterns for fixed deferrable and fixed non-deferrable isnot needed in optimization formulation. Instead, inventors havedecoupled the complicated and multi-objective optimization problem intosmall, independent optimization sub-problems, which allows theoptimization modules to generate DR policies for regulatable and fixeddeferrable loads more effectively and efficiently.

In particular, in Equation 10 below, fixed loads are assumed to befuture constant loads although it may not be known exactly how muchelectric energy a fixed load will use for the next day. This futurefixed load concept may be applied to energy generated from renewablesources as well (such as solar PVs or wind turbines) except that theenergy consumption from a renewable source is negative instead ofpositive. Hence, removing the fixed loads and fixed renewable sourcesfrom the objective function does not affect the optimal solution so thatan equivalent and simpler optimization problem may be obtained, which isdescribed by

$\begin{matrix}{{{{Minimize}\text{:}\mspace{14mu} C} = {\sum\limits_{i}\; {\left( {p_{i} - p_{i}^{PV} - p_{i}^{batt}} \right) \cdot \left( {Q_{i}^{HVAC} + Q_{i}^{deferrable}} \right)}}}{{{{Subject}\mspace{14mu} {to}\text{:}\mspace{14mu} 0} \leq Q_{i}^{HVAC} \leq Q_{\max}},{T_{\min} \leq T_{i}^{I} \leq T_{\max}}}} & (10)\end{matrix}$

Also, unlike the HVAC load, a fixed deferrable load has a fixed energyconsumption pattern. This fixed load pattern can be applied to differenttime frames. Assuming that the fixed load pattern is Q^(deferrable)within one-hour time frame, then Equation 10 can be simplified as

$\begin{matrix}\left\{ {\begin{matrix}{{{Minimize}\text{:}\mspace{14mu} C_{HVAC}} = {\sum\limits_{i}\; {\left( {p_{i} - p_{i}^{PV} - p_{i}^{batt}} \right) \cdot Q_{i}^{HVAC}}}} \\{{{{Subject}\mspace{14mu} {to}\text{:}\mspace{14mu} 0} \leq Q_{i}^{HVAC} \leq Q^{\max}},{T^{\min} \leq T_{i} \leq T^{\max}}}\end{matrix} + \left\{ {{{Minimize}\text{:}\mspace{14mu} C_{deferrable}} = {\sum\limits_{i}\; {\left( {p_{i} - p_{i}^{PV} - p_{i}^{batt}} \right) \cdot \theta_{i} \cdot Q^{deferrable}}}} \right.} \right. & (11)\end{matrix}$

where θ_(i) is a binary integer array to indicate the status of thedeferrable load (e.g., on or off). For example, if the deferrable loadis operated at ith hour of the day for one hour, then,

$\theta_{i} = {\begin{pmatrix}0 & \ldots & 0 & \underset{\underset{{ith}\mspace{11mu} {hour}}{\wr}}{1} & 0 & \ldots & 0\end{pmatrix}^{T}.}$

Equation 11 indicates that the integrative optimization problem ofEquation 10 is actually equivalent to two independent optimizationproblems. In summary, the optimization and energy management problem forfixed non-deferrable loads, regulatable loads, and fixed deferrableloads can be solved separately and then combined together to achieve thefinal optimal solution. Note that loads in each category are alsoindependent. Therefore, a complicated and multi-objective optimizationproblem is decoupled into several small independent optimizationproblems that are used to generate robust optimal energy managementpolicy and decisions effectively and efficiently. Energy consuming unitshaving future fixed non-deferrable loads may be turned off via one ormore sensors when not needed. For example, as shown in FIG. 20, lightsin a room may be turned on in response to one or more sensors detectingthe presence of a person in the room and turned off in response to oneor more sensors detecting that the person has left the room (either viano input to that sensor or via input received by a sensor in anotherroom).

The DR strategy for regulatable loads may be based on the modelednext-day energy consumption and day-ahead energy pricing. For example,for the HVAC unit, the optimization module uses the HVAC energyconsumption model developed by the learning module to estimate HVACenergy consumption in each time window for the next day (e.g., 24 hours)and creates a DR strategy for the HVAC unit based on the modeledconsumption, the weather forecast, and the next day energy prices. TheDR strategy for the HVAC unit includes a plurality of thermostatsettings corresponding to each time window for the next day that are themost cost-effective settings for the next day. Because the model isupdated daily, an optimal and substantially real-time DR strategy thatis solved according to the learning-based adaptive model results in moreefficient operation of a HVAC unit for different seasons, users, andweather conditions. The optimization module may also be configured foradjusting the DR strategy for the current day based on RTP data for thecurrent day. Initially, the queues used by the learning module do notcontain sufficient amounts of data for the learning module to use alearning based approach to model energy consumption. Until the queuesare sufficiently full, the optimization module may use an optimizationapproach based on a simplified energy consumption model, such as shownin Equation 5, and DAP data or a heuristic approach (see FIG. 1) and RTPdata to generate a DR strategy for one or more energy consuming units.

For fixed deferrable loads, such as the dishwasher or dryer, the DRstrategy may be based on real time energy pricing and acceptable priceranges and/or time frames during which the energy consuming unit may beoperated. For these energy consuming units, the optimization module mayreceive from the customer an indication that the unit is operable whenthe RTP is lower than a predefined price set by the customer. Thisindication may be subject to operation during an acceptable time frame.Otherwise, those energy consuming units operate at the time frame thathas the lowest price within a customer preferred time window. Bydelaying the operation of these energy consuming units, the total energyconsumption of the house does not change. However, delaying operation toa non-peak or lower price time period may reduce the peak load anddecrease the overall energy cost for the day, depending on the pricesfor the day.

For example, when DR strategies are not used, it may be assumed that thedishwasher operates between 7 pm to 8 pm and the dryer operates between6 pm to 9 pm on Monday, Wednesday, and Friday. On Sunday, 50% usage ofdishwasher and dryer is defined between 7 pm to 8 pm and 6 pm to 9 pm,respectively, for consideration of general light load usage of thoseappliances during the weekend.

Using a DR strategy, the customer may specify instead that thedishwasher may operate for an hour or less one time every two days, forexample, but the customer may not have a preference as to when it isoperated so long as the price of electricity during operation is withina particular range set by the customer. The operation schedule and/orpreferred pricing ranges may be input by or at the direction of thecustomer, for example. In other implementations, the usage schedule maybe set by another entity, such as the utility company, or set by theHEMS 10 based on DTP, RTP, energy generated by renewable sourcesassociated with the house, and/or expected usage of other energyconsuming units in the house.

Exemplary Flow of the HEMS 10

FIGS. 5A and 5B illustrate an exemplary flow 100 of the HEMS 10 forgenerating a DR strategy for an HVAC unit according to oneimplementation. If actual data is not available upon start up of thesystem 10, the system 10 establishes an initial DR strategy to use untilsufficient initial data is collected and stored in the queues 26. Steps101 through 113 below describe exemplary steps for establishing aninitial DR strategy until sufficient amounts of actual measured data areavailable. The system 10 may use a simplified DR strategy, such as aheuristic DR method (FIG. 1), to run the HVAC unit until sufficientmeasured energy consumption data of the HVAC unit is stored in thequeues 26. For a new home or HVAC unit, the data queues are empty at thebeginning, so an optimization routine cannot be developed and applied.Thus, initial DR policies may be generated by using a simplified DRmechanism.

Steps 115 through 127 describe exemplary steps for establishing a moreaccurate DR strategy once sufficient data is stored. After the dataqueues 26 are full, the learning module learns the HVAC energyconsumption model based on the measured energy consumption data storedin the queues 26, and the optimization module generates optimal DRpolicy everyday based on the learned model, weather forecast, andforecasted electricity pricing (DAP). The learning and optimizationmodules may be implemented via a substantially real-time computingsystem, such as digital signal processing (DSP) chips, by usingassembly-language-based software.

Referring back to FIG. 5A, beginning at Step 101, DAP data and weatherforecast for a location of the building are received by the system 50,such as by the communication module. For example, 24-hour electricityDAP data can be provided by an electric utility one day ahead under adynamic pricing framework. The energy pricing data for electricity istypically provided by the electric utility company. Energy pricing datafor natural gas is typically provided by the natural gas company. Inaddition, 24-hour weather forecast data may be available from NationalWeather Service, for example. The weather forecast data for the locationmay include 24-hour next-day weather forecast data and current-dayweather data, such as predicted and current hourly temperatures,humidity, and wind speed and direction.

In Step 103, an estimated usage schedule for the HVAC unit is received.The estimated usage schedule includes, for example, when and for howlong the HVAC is operated and how much energy is needed to operate it.For example, the HVAC unit may be operated throughout the day dependingon the indoor temperature, and the length of time for operation and theamount of energy needed for operation depends on the outdoortemperature, the thermal characteristics of the house, and theefficiency of the HVAC unit.

A next-day DR strategy is created in Step 107 by using a simplified DRtechnique, such as 1) an optimal HVAC DR policy generated based on thesimplified energy consumption model (Equation 5) and DAP data or 2) anHVAC DR policy generated by using a heuristic approach based on DAP data(FIG. 1). The next-day DR strategy sets forth the operating parametersof one or more energy consuming units at different times of the day. Forexample, the DR strategy may set forth the hourly thermostat settingsfor the HVAC system and time frames for operation of the dishwasher,washer, and/or dryer.

Referring back to the implementation shown in FIG. 5A, if RTP data isavailable for the location, the RTP data may be received in Step 108,and the heuristic approach or the simplified optimization approach maybe used with the next-day DR strategy and RTP data to adjust the DRstrategy during the current day in which the DR strategy is applied tooptimize the energy costs for the customer, as shown in Step 109. Thisstep may be omitted, however. For example, Steps 108 and 109 may beomitted if RTP data is not available for the location or if there areother incentives given to customers by the utility company. In analternative implementation (not shown), the RTP data may be used toevaluate the effectiveness of the DR strategy for the current day inaddition to or in lieu of Step 109. Actual energy consumption data,indoor temperatures, outdoor temperatures, and thermostat settings foreach time window (e.g., each hour or preset time frame) during a day arestored in the memory in respective queues 26, as shown in Step 111, bythe data management module. The data is stored in each queue on a FIFObasis for a certain number of days n. For example, in oneimplementation, the data may be stored for 8 days. Thus, when new datais received each day, the oldest data saved in the front of the queuesis removed and the new data is saved. Maintaining the queue on a FIFObasis provides up-to-date household energy consumption data for thelearning module, which allows the learning module to more accuratelycapture the thermal and appliance energy usage behavior of a house fordifferent seasons, users, and weather conditions, for example. Thenumber of days for which the queues are configured for storing data maybe selected such that any update of the HVAC energy consumption based onthe data saved in the queues should reflect the impact of the seasons,weather, users, and house conditions, for example, within one or twoweeks.

The system repeats Steps 101 through 111 until the memory queues 26 arefull, as shown in Step 113. Alternatively, if a data acquisition orstorage component of the system 10 has a problem that prevents storageof data for one or more days into the queues 26, Steps 101 through 113may be executed until the model is stabilized (e.g., the queues 26 arefully populated by data that can be used by the learning module).

Once the memory queues 26 are full, DAP data and weather forecast datafor the location are received in Step 115, such as by the communicationmodule. Then, in Step 117, the learning module models the energyconsumption for the next day using a learning-based approach, such asneural network learning or regression-based learning, which aredescribed above in the section entitled “Learning Module.” The modeledenergy consumption is based on the weather forecast data and actualusage data stored in the memory queues 26, for example. In oneimplementation (not shown), the learning module may be configured forreceiving a selection from the user regarding which learning basedapproach the user prefers for the system 10 to use. In anotherimplementation (not shown), the user may be able to select a particularlearning based approach for the system 10 prior to or duringinstallation of the system 10.

In Step 119, a DR strategy is created for the next day by theoptimization module based on the energy consumption modeled in Step 117and the DAP data received in Step 115. This DR strategy is communicatedto the HVAC unit in Step 126 by the communication module.

In Step 127, RTP data and energy usage data for the current day may bereceived by the communication module, and in Step 121, the DR strategythat was created the day prior for the current day may be adjustedduring the current day based on the usage data and/or RTP data receivedfor the current day. However, if RTP data is not available for thelocation or if there are other incentives given to customers by theutility company, these steps may be omitted.

In Step 123, the data management module removes usage data from thequeues for the oldest day and stores usage data for the most recent day(e.g., the current day or one day prior) in the queues. Then, Steps 115through 127 are repeated for the next day.

The various modules described above in FIGS. 4 and 5A-5B are exemplaryand the functions described as being performed by each module may beperformed by other modules executed on the same or another computingdevice.

Except for the queues, it is possible to use other data structures, suchas trees, tables, or database management systems, to store the weatherand usage data.

FIGS. 5A and 5B illustrate an exemplary process for generating a DRstrategy for the HVAC unit, but the process and data described above maybe adapted for other types of regulatable energy units.

Algorithm 1 below presents an exemplary pseudo-code that illustrates howan HVAC energy consumption model is learned and how an optimal DR policyfor operating the HVAC unit is generated and updated. Lines 1 to 8correspond to Steps 101 through 113 of FIG. 5A for establishing aninitial DR strategy until sufficient amounts of actual data areavailable.

Lines 9 and 10 correspond to Steps 115 through 127 of FIG. 5B forestablishing a more accurate DR strategy once sufficient data is stored.A HVAC energy consumption model for the time window represented by thelength of the queues is obtained through the regression or neuralnetwork approach, based on which an optimal DR policy is generated (line10). The DR policy is loaded into a programmable thermostat to controlHVAC energy consumption for the next day. During the operation of theHVAC unit the next day, the actual measured energy consumption results,together with the actual measured weather information, 24-hourthermostat settings and actual measured 24-hour indoor room temperature,are saved in the queues. At the same time, old 24-hour data in the frontof the queues are removed. The process continues to update HVAC energyconsumption model and generate new optimal DR policy day after day.

Algorithm 1: Learning-based demand response 1: {Initial demand responseSteps 101 through 113as shown in FIG. 5A} 2: for d=1 to numDay 3: T_(queue) ^(O) ← T_(i) ^(O) , P_(queue) ← P_(i) (i=1,......, 24) 4: Obtain 24-hour thermostat settingT_(i)(i=1, ..., 24)  (Step 107 asshown by Fig. 5A). 5:  Programmable thermostat 

 T_(i)(i=1, ..., 24). 6:  Obtain next-day HVAC energy consumption Q_(i)(i=1, ..., 24) through real-time measurement 7:  Q_(queue) ←Q_(i), T_(queue) ← T_(i)(i=1, ..., 24) 8: end for 9: {Updating HVACenergy consumption model and generating optimal DR policy (Steps 115through 127 as shown by FIG. 5B) using nonlinear programming or PSOmethod (see discussion of optimization module below) } 10: Runningalgorithm day after day (go back to line 9).

Algorithm 2 shown below provides the pseudo-code of the particle swarmoptimization (PSO) method according to one implementation. In PSO, eachsingle solution is called a particle in the search space. All ofparticles have fitness values which are evaluated by the fitnessfunction to be optimized, and have velocities which direct the flying ofthe particles. Each candidate solution can be thought of as a particle“flying” through the fitness space by updating the candidate velocityand position based on both global best fitness position and local bestfitness position.

Algorithm 2: PSO during the iteration of Alg. 1 in line 9 1: Initializeparticle position: T_(buf) (m) ∈ [T^(min),T^(max) ] 2: Initializeparticle speed: {tilde over (T)}_(buf)(m) ∈[−ΔT,ΔT], m=1,...,M 3:{Calculate initial fitness values for all particles}fitness(m)=f(T_(buf)(m)), m = 1,...,M 4: {circumflex over (T)}_(G) ←T(k); if fitness(k)=max{fitness(m),m∈ [1, M]} {circumflex over (T)}(m) ←T(m); 5: do 6:  {Update velocity for all particles}  {tilde over (T)}(m)= w{tilde over (T)}_(buf)(m)+c₁ · rand(0,1)[{circumflex over(T)}(m)−T_(buf)(m)]    +c2 · rand(0,1)[{circumflex over (T)}_(G)−T_(buf)(m)] 7:  {Update position for all particles}  T(m) = T_(buf) (m) +{circumflex over (T)}(m) 8:  if T(m) out of boundary → boundary handlingT(m) 9:  {Calculate fitness values for all particles}  fitness(m)=f(T(m)), m = 1,... ,M 10:  {circumflex over (T)}_(G) ← T(k); iffitness(k)=max{fitness(m), m ∈ [1,M]}  {circumflex over (T)}(m) ← T(m);if T(m) > T_(buf)(m) 11:  T_(buf)(m)←T(m); {tilde over(T)}_(buf)(m)←{tilde over (T)}(m) 12: while maximum iterations or a stopcriteria is not reached 13: Output global optimal solution {circumflexover (T)}_(G) to Alg. 1.

In Alg. 2, T(m), the position of a particle, represents a 24-hourthermostat setting during a day. {tilde over (T)}(m), the speed of aparticle, represents the thermostat adjustment during one iteration ofthe PSO algorithm. {circumflex over (T)}(m) and {circumflex over(T)}_(G) are the individual best thermostat setting associated withparticle m and the global best thermostat setting of all the particles,respectively. In line 6, c₁ and c₂ are the user defined coefficients,and w is the inertia weight used to balance global and local search.These parameters are determined through trial and error until a bestpossible result is obtained. For each updated particle positioncalculated in line 7, it is checked in line 8 whether the updatedposition is out of the boundary. If so, for any temperature settingbeyond [T^(min), T^(max)] it is reset to T^(min) or T^(max) depending onwhether the temperature setting is smaller than T^(min) or larger thanT^(max) [40]. The fitness function ƒ(•) is defined by

$\begin{matrix}{{f\left( T_{m} \right)} = \left\{ \begin{matrix}{{1/\left\lbrack {\sum\limits_{i = 1}^{24}\; {p_{i} \cdot {Q(m)}_{i}}} \right\rbrack},} & {Q^{\min} \leq {Q(m)}_{i} \leq Q^{\max}} \\0 & {else}\end{matrix} \right.} & (12)\end{matrix}$

where Q(m)_(i)=q(T_(i+1),T_(i),T_(i) ^(O),β) is the energy consumptionof the HVAC for particle m at ith hour (i=1, . . . , 24), and Q^(min)and Q^(max) stand for the minimum and maximum HVAC energy consumptioncorresponding to a practical HVAC unit.

The global market for grid-connected residential solar PV installationscoupled with energy storage is predicted to grow tenfold to reach morethan 900 megawatts in 2018, making smart management and control ofresidential PV systems and energy storage increasingly important. Toelectric utilities, proper management of PV and energy storage will helpto maintain grid reliability, resiliency, and power quality whileminimizing curtailment of available solar power. To home owners,effective use of PV and energy storage will help consumers to save moneyoff the electricity bill. Residential PV and energy storage that ismanaged and controlled holistically according to dynamic electricityprices and tariffs in a demand response (DR) framework may benefit theutility company and residents. Based on this framework, the HEMSprovides the most efficient and cost effective grid integration ofresidential energy storage, solar PVs, and home appliances, according tovarious implementations.

For example, FIGS. 16 and 17 illustrate diagrams of a residence in whichsolar PVs, home appliances, and residential energy storage areintegrated into the HEMS 10. In particular, the HEMS 10 according to theimplementation shown in FIG. 17 further includes an inverter 4, abattery 7 for storing energy generated by the PV system 1 anddischarging energy to energy consuming units within the building, asupercapacitor in electrical communication with the battery 7 thatabsorbs most of the large and fast ramps in PV power output (e.g.,caused by cloud movement). The PV system 1 may be configured foroperating in maximum power point tracking (MPPT) mode, according to oneimplementation. The HAS 14, or another controller in electricalcommunication with the HEMS 10, may be further configured for: (1) usingartificial neural networks (ANNs) and adaptive dynamic programming (ADP)to control the inverter 4, (2) identifying when power should be releasedor stored by the battery by integrating real-time dynamic impedancemeasurements for battery charging/discharging algorithms, and (4)coordinating usage of regulatable and deferrable energy consuming unitswith PV and energy storage.

Security System for HEMS

Various implementations of the HEMS 10 may also include a securitysystem that assures that DR policies generated one day or hours aheadare executed next day or later without modification. The security systemmay communicate with a smart meter that is configured for proving thecorrectness of any smart appliance, a group of smart appliances that areconfigured for proving the correctness of the smart meter, and a serviceprovider configured for proving the correctness of the smart meter. Inone exemplary security system, each meter has a copy of its own log, andeach household meter is mapped to several other witnesses. The systemalso includes a commitment protocol that is used to ensure thatwitnesses will retrieve exactly the same log as the observation objectowns. In addition, the system includes a challenge/response protocol toaddress the problem that some household meters do not response or failto acknowledge whether messages were sent successfully or not.

The security system maintains the confidentiality, availability, andintegrity of information and systems and methods of implementing theabove-described functionality and other smart home functions. Inaddition, accountability mechanisms may be also be included that providetrustworthy smart appliances in homes and overcome the challenges forintegration of physical systems and human factors. Thus, the securitysystem may be configured for (1) accountable, non-repudiable, maliciousappliance inspection, (2) denial of service (DoS) attack mitigation foroperation of diverse DR appliances and renewable sources in the smarthome area, and (3) interoperability of energy system, wirelesscommunication, and security methods.

The accountability mechanisms may be configured for verifying thecustomer responses and the smart appliance group and for adopting logicto provide theoretical proofs of accountability for the designedprotocols. The security mechanisms and protocols may also allow autility company to prove the correctness of DR policies actually used tooperate home appliances.

The designed protocols may achieve the following exemplary goals: (a) asmart meter that can prove the correctness of smart appliances in ahome; (b) a group of smart appliances that can prove the correctness ofthe smart meter; and (c) a service provider that can prove thecorrectness of the smart meter. The witness mechanisms may includeseveral protocols. For example, one protocol may be that every meter hasone copy of its own log, which is ensured by a tamper-evident, logmechanism. Other logs are retrieved when required. Meters exchange justenough messages to prove themselves. As another example, each householdmeter is mapped to several other witnesses. Each witness collects itslog, check its correctness by comparing the readings, and reports theresults to the rest of the system. The witnesses are reassignedobservation objects according to function w at set intervals. Anotherexample includes a commitment protocol that is used to ensure thatwitnesses will retrieve exactly the same log, as the observation objectowns. It also guarantees that no one can deny a received message.Furthermore, another example includes a challenge/response protocol toaddress the problem that some household meters do not respond or fail toacknowledge whether messages were successfully sent or not.

Mutual-inspection and malicious appliance inspection mechanisms may alsobe included. Mutual-inspection mechanisms may realize non-repudiation insmart homes so that any misbehavior and malicious operation compromisingpower reading is eventually detected either by the utility company or byindependent users. The system may also jamming mitigation techniques,sampling synchronization, and how to prevent wireless network delay andwireless jamming. The malicious appliance inspection problem may beaddressed via a suite of tree-based inspection algorithms (including anadaptive tree approach) for both static inspection and dynamicinspection, which are investigated through theoretical analysis,simulations, and experiments.

Second, to prevent the HEMS wireless networks from being jammedmaliciously to cause DoS attacks, the security system may include DoSattack mitigation methods for diverse DR appliances and renewablesources in the smart home area. For example, these methods may includechannel hopping when a DoS attack happens, such as rendezvous channelhopping algorithms via catalan number series, as channel hoppingpatterns with adjustments.

Third, interoperability and security are tightly coupled activities, andthese joint activities satisfy multiple requirements in both domains,including interoperability requirements for transparency and clarity insystem interfaces and relationships, and security requirements forresistance to attack, self-checking, system access control, etc.

Computational Experiment System for Comparing DR Strategies Based onVarious Energy Consumption Models

A great difficulty for DR study is how to investigate and evaluate theperformance of different DR strategies because such study requires theDR strategies to be applied to the same house condition. For a practicalresidential house, it is impractical to meet this requirement.

DR strategies developed using various learning-based modeling approachesmay be compared, researched, and evaluated by using a computationalexperiment system that simulates energy consumption of a real-lifehouse. This section describes an exemplary computational experimentsystem for comparing DR strategies based on various energy consumptionmodels. The computational experiment system 60 shown in FIGS. 7 and 9 issimilar to the real-life DR strategy system 10 described above inrelation to FIGS. 2-5B except that the system 60 uses simulated energyconsumption data generated by building simulation software instead ofactual energy consumption data, according to various implementations. Inaddition, the system 60 may generate multiple DR strategies based onvarious energy consumption modeling approaches so that the cost andenergy effectiveness of the multiple DR strategies may be compared. Inone implementation, this system 60 may provide information to the enduser comparing the energy costs and/or consumption resulting from use ofthe DR strategy system 10 and the costs and consumption resulting fromanother type of DR system or from not using a DR strategy.

To accomplish the computational experiment system, MATLAB, a powerfultechnical software system, is integrated with building simulationsoftware for the DR evaluation, according to one implementation. Theintegrated simulation strategy may greatly accelerate the offline designand assessment of a DR strategy. Other suitable software can also beused for such offline simulation evaluation.

The building simulation software, such as eQUEST, Energy Plus andothers, is used as a virtual test bed to simulate home energyconsumption. The building simulation software allows one to “build” asimulated house that is similar to a practical one. It uses standardcommercial building materials defined in the software library andreal-life weather and solar data, which may be obtained from the USDepartment of Energy, for example. Thus, the building simulationsoftware is capable of estimating changes in the electrical load of a“practical” house throughout a year, certain days within the year, orcertain time period of a day.

To simulate energy consumption of a residential house, an architecturalmodel of the house is created based on the blueprint and constructionmaterials used to build the house. For example, a generic floor plan fora two story, 2,500 square foot house may be used with a locationselection of Springfield, Ill. Next, internal loads are defined. Theinternal loads may include, for example, washing machines, dryer, waterheater, dishwasher, electric stove, refrigerator, microwave oven, lightsfor each room, occupants, etc. Then, appropriate schedules are input tomodel these loads, such as when, how long, and how much of these loadsare used each day. The final step is defining the HVAC system. In thisexample, a three-ton unit is selected to condition the first floor and atwo-ton unit is selected to condition the second floor. Additionaldetails about how to build a simulated house can be found in the eQuestIntroductory Tutorial, version 3.64 by James J. Hirsch & Associates,December 2010, for example. The simulation software uses the aboveinformation to simulate the amount of energy consumed during varioustime windows of a day (e.g., hourly) based on the actual weather datafor that day for the specified location.

FIG. 7 illustrates a schematic diagram of a central server 600 forimplementing a computational experiment system 60 according to oneimplementation, and FIGS. 8A-9 illustrate an exemplary flow for thesystem 60 according to one implementation.

As shown in FIG. 7, the central server 600 may include a processor 610that communicates with other elements within the central server 600 viaa system interface or bus 645. Also included in the central server 600may be a display device/input device 620 for receiving and displayingdata. This display device/input device 620 may be, for example, akeyboard or pointing device that is used in combination with a monitor.The central server 600 may further include memory 605, which may includeboth read only memory (ROM) 635 and random access memory (RAM) 630. Theserver's ROM 635 may be used to store a basic input/output system 640(BIOS), containing the basic routines that help to transfer informationacross the one or more networks.

In addition, the central server 600 may include at least one storagedevice 615, such as a hard disk drive, a floppy disk drive, a CD Romdrive, or optical disk drive, for storing information on variouscomputer-readable media, such as a hard disk, a removable magnetic disk,or a CD-ROM disk. As will be appreciated by one of ordinary skill in theart, each of these storage devices 615 may be connected to the systembus 645 by an appropriate interface. The storage devices 615 and theirassociated computer-readable media may provide nonvolatile storage for acentral server. It is important to note that the computer-readable mediadescribed above could be replaced by any other type of computer-readablemedia known in the art. Such media include, for example, magneticcassettes, flash memory cards and digital video disks.

A number of program modules may be stored by the various storage devicesand within RAM 630. Such program modules may include an operating system650 and a plurality of one or more (N) modules 660. The modules 660 maycontrol certain aspects of the operation of the central server 600, withthe assistance of the processor 610 and the operating system 650. Forexample, the modules may perform the same or similar functions describedabove in relation to FIGS. 4 and 5A-5B and below in relation to FIGS.8A-8B and illustrated by the figures and other materials disclosedherein, such as executing various functions of the computationalexperiment system 60.

In one exemplary implementation, the server 600 includes the followingmodules: a simulation module, a learning module, an optimization module,a data management module, and a pricing module.

The simulation module uses the next-day DR strategy determined the daybefore by the optimization module for the system 60 and current dayweather data to simulate the energy consumption of the house for thecurrent day. In particular, energy consumption of a residential house issimulated by using the simulation module of the computational experimentsystem 60 for a practical weather pattern of the day, includingtemperature, humidity, solar radiation, etc., at a location. The resultsgenerated by the simulation module of system 60 are stored in the queuesand are received by the learning module of system 60 to update themodeled energy consumption each day.

The learning, optimization, and data management modules of thecomputational experiment system 60 are similar to the correspondingmodules described above for the DR strategy system 10, except that thequeues include simulated energy consumption data generated by thesimulation module instead of actual energy consumption data. FIG. 9illustrates the schematic flow of data through the queues and thelearning, optimization, and simulation modules of the system 60.

The pricing module calculates electricity pricing for the current daybased on RTP data and the simulated energy consumption for the house.The pricing module may also calculate energy pricing for the next daybased on DAP data and the simulated energy consumption for the next day.For an implementation in which two or more learning approaches are usedto model the energy consumption of the house and a DR strategy isdetermined based on each learning approach, the pricing module maygenerate one or more displays indicating the calculated prices forenergy usage over the course of a day using each DR strategy, such asthe graphs shown in FIGS. 11-15.

FIGS. 8A-8B illustrate an exemplary flow 200 of the computationalexperiment system 60 according to one implementation. Beginning at Step201 in FIG. 8A, the DAP pricing data and weather forecast data arereceived for a particular location. In Step 203, the estimated usageschedule for an HVAC unit for the house for the next day is received. InStep 205, a next-day DR strategy is created by the optimization moduleby using a simplified DR technique, such as (1) the optimal HVAC DRapproach based on the simplified energy consumption model shown inEquation 5 and DAP data or (2) a heuristic approach based on DAP data(FIG. 1). Then, in Step 207, the DR strategy is communicated to thesimulation module to simulate energy consumption for the next day basedon the DR strategy and actual weather for that day. If RTP data isavailable for the location, the RTP data and the simulated energy usagefor the HVAC unit may be received for the current day, as shown in Step208, and the RTP is used to adjust the DR strategy during the day it isbeing applied, as shown in Step 211.

In Step 209, the simulated energy consumption data, the thermostatsettings, and the temperature data (indoor and outdoor) for each timewindow for the current day are stored in the memory queues.

In Step 213, the system 60 determines whether the queues are full. Ifthe queues are not full, then Steps 201 through 211 are repeated for thenext day. If the queues are full, then the system 60 proceeds to Step215 in FIG. 8B in which DAP and weather forecast data are received forthe next day. In Step 217, the learning module generates at least oneenergy consumption model for the HVAC unit using one or morelearning-based approaches based on the weather forecast, thermostatsettings, and the simulated energy consumption data stored in thequeues. In Step 219, the optimization module generates one or more DRstrategies based on the DAP, the weather forecast, and the one or moreenergy consumption models from Step 217. The DR strategies may be basedon the same modeling approach or different modeling approaches.Alternatively, or in addition thereto, the DR strategies may be solvedusing the heuristic approach, the optimal approach from Equation 5 orEquation 8, or the PSO approach using Equation 4 or Equation 8. In Step221, the DR strategies are communicated to the simulation module tosimulate energy consumption for the next day based on the actualweather. In Step 223, the simulated energy data from the simulationmodule is communicated to the data management module for storing thedata in the queue, and old data is removed. In addition, in Step 225,RTP data, if available, is received along with simulated energy usagedata for the HVAC unit for the current day. Then, in Step 227, the RTPdata is used to adjust the DR strategy during the day it is applied.Steps 215 through 227 are then repeated each day. The energy costsand/or usage associated with each DR strategy are displayed by thepricing module (not shown), which allows a user to compare DR strategiesand which yield lower energy costs and usage.

FIGS. 8A and 8B illustrate an exemplary process for generating one ormore DR strategies for the HVAC unit, but the process and data describedabove may be adapted for other types of energy consuming units.

The exemplary pseudo code in Alg. 1 above may be adapted for use withthe simulated energy consumption data from the simulation module tomodel energy consumption and generate a DR policy for the next day.Lines 1 to 8 correspond to Steps 201 through 213 of FIG. 8A forestablishing an initial DR strategy until sufficient amounts of actualdata are available. Lines 9 and 10 correspond to Steps 215 through 227of FIG. 8B for establishing a more accurate DR strategy once sufficientdata is stored.

FIG. 10 compares HVAC energy consumption models created using thesimplified approach, the neural network approach, and the regressionapproach with the energy consumption simulated by the simulation moduleof system 60, which is indicated as “eQuest” in FIG. 10. For the neuralnetwork and regression approaches, the model is obtained based on thedata saved in the queues, updated each time when new data is added intothe queues, and then used for the energy consumption estimation for thenext day. For example, if the model is updated on Tuesday, it will beused for energy consumption estimation on Wednesday. As shown in FIG.10, results generated by using the simplified model are quite differentfrom actual simulated energy consumption. However, for neural networkand regression approaches, the estimation becomes more reliable andaccurate as more data are collected and used to learn the model. FIG.10( a) illustrates the modeled energy consumption using two days ofdata, 10(b) illustrates modeled energy consumption using four days ofdata, and 10(c) illustrates modeled energy consumption using six days ofdata. As shown in these three graphs, the difference between the modelslearned by using the neural network and regression approaches is small,and the estimation by the neural network and regression approachesbecomes closer to the actual simulated energy consumption as data frommore days are stored in the queues.

To better understand the performance, a comparison study is made asshown by FIGS. 11 and 12 through the computational experiment system 60,in which “None” represents a constant thermostat setting at 72°,“Heuristic” stands for the thermostat setting by using heuristic DRstrategy, “Simple” signifies the optimal thermostat setting based on thesimplified model (see Equation 5), and “Regression” represents theoptimal thermostat setting by using a substantially real-time adaptiveregression model (See Equation 8). For the heuristic algorithm, a“nine-point” thermostat setting at 71°; 72°, 73°, 74°, 75°, 76°, 77°,78° and 79° is used, in which 71° and 79° are the thermostat settingscorresponding to PR^(min) and PR^(max) during a day, respectively. Forthe optimal DR strategy algorithms, the upper and lower temperaturesettings are 71° and 79°, respectively. Table 1 below shows a comparisonof the HVAC cost for five consecutive days, in which theregression-based approach updates the HVAC energy consumption modeldaily when new energy consumption data is available. FIG. 12 shows acomparison of the HVAC cost for one month by using different energyconsumption models based on DAP tariff. As can be seen from FIGS. 11 and12 and Table 1, as well as other results, the learning-based DRstrategies are the most efficient among all the DR strategies,demonstrating the effectiveness and excellent performance of thelearning based DR strategy systems.

TABLE 1 HVAC energy cost for five consecutive days based on DAP Day 1Day 2 Day 3 Day 4 Day 5 None $1.571 $1.503 $1.181 $0.867 $1.571Heuristic $1.124 $1.028 $0.850 $0.687 $1.020 Approach Simple $1.019$0.997 $0.795 $0.642 $0.983 Optimization Approach Regression $1.015$0.988 $0.789 $0.595 $0.902 Based Approach

Table 2 below shows a comparison of the HVAC cost for a high, mild, andlow cost day for RTP and DAP tariffs. PSO and regression-basedapproaches are listed separately.

TABLE 2 HVAC energy cost for five continuous days Highest Mild Low RTPDAP RTP DAP RTP DAP None $5.09 $3.44 $1.60 $1.57 $1.36 $1.32 HeuristicApproach $3.77 $2.28 $1.08 $1.38 $1.04 $0.88 Optimization $3.12 $2.05$0.93 $0.80 $0.90 $0.82 Approach based on Eq. (5) Optimization $3.12$1.98 $0.73 $0.76 $0.73 $0.66 Approach based on Eq. (8) PSO $3.11 $1.94$0.77 $0.56 $0.88 $0.76 Approach based on Eq. (8)

FIG. 13 compares total energy and energy cost for the house with andwithout demand response during a high DAP day on a Friday in July. Asshown in FIG. 13, with demand response, peak load is clearly reduced andthe energy usage of dryers and dishwasher is shifted to low cost timeperiod of the day.

Compared to conventional simplified models, the learning-basedapproaches to modeling home energy consumption can accurately capturethe energy consumption of a house under complicated weather conditions.In particular, FIGS. 14 and 15 illustrate a comparison of the HVACenergy consumption and cost using the following DR strategy approaches:none, a heuristic approach (“Heuristic”), a simplified optimal approach(“Model-OP”) based on Equation 5, an optimal approach based on regressedenergy consumption model (“Reg-OP”), and a PSO optimization approachbased on regressed energy consumption model (“PSO-OP”). FIG. 14illustrates the comparison using RTP tariff for the highest RTP tariffday in the summer, and FIG. 15 illustrates the comparison using the DAPtariff for the highest DAP tariff in the summer. The comparisons inFIGS. 14 and 15 show that the difference in cost saving between the DRstrategies solved by using normal nonlinear programming technique(“Reg-OP”) and the PSO method (“PSO-OP”) is small.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousimplementations of the present invention. In this regard, each block inthe flowchart or block diagrams may represent a module, segment, orportion of code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theimplementation was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious implementations with various modifications as are suited to theparticular use contemplated.

Any combination of one or more computer readable medium(s) may be usedto implement the systems and methods described hereinabove. The computerreadable medium may be a computer readable signal medium or a computerreadable storage medium. A computer readable storage medium may be, forexample, but not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, ordevice, or any suitable combination of the foregoing. More specificexamples (a non-exhaustive list) of the computer readable storage mediumwould include the following: an electrical connection having one or morewires, a portable computer diskette, a hard disk, a random access memory(RAM), a read-only memory (ROM), an erasable programmable read-onlymemory (EPROM or Flash memory), an optical fiber, a portable compactdisc read-only memory (CD-ROM), an optical storage device, a magneticstorage device, or any suitable combination of the foregoing. In thecontext of this document, a computer readable storage medium may be anytangible medium that can contain, or store a program for use by or inconnection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to implementations ofthe invention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

1. An energy consumption management system comprising: one or more localreceivers disposed adjacent a respective one of one or more energyconsuming units in a building, each local receiver comprising aprocessor configured for receiving usage instructions for the adjacentenergy consuming unit and causing the usage instructions to be executedfor the energy consuming unit; and a central computing systemcomprising: a memory configured for storing actual usage data associatedwith at least one energy consuming unit in the building for one or moretime windows on each of days i through j, wherein day i is the first dayfor which data is stored and day j is the most recent day for which datais stored; and a processor configured for: receiving the actual usagedata from the memory; executing at least one computer-based learningsystem to model energy consumption for day j+1 based on at least theactual energy usage data for the energy consumption unit; generating ademand response strategy for the energy consuming unit for day j+1 basedon the modeled energy consumption and next-day energy pricing for eachtime window for day j+1; and communicating the demand response strategyof the energy consuming unit to the local receiver associated with theenergy consuming unit, the demand response strategy comprising the usageinstructions.
 2. The system of claim 1, wherein the computer-basedlearning system is a neural network system.
 3. The system of claim 1,wherein the computer-based learning system is a regression based system.4. The system of claim 1, wherein causing the usage instructions to beexecuted for the energy consuming unit comprises communicating the usageinstructions to the energy consuming unit.
 5. The system of claim 1,wherein causing the usage instructions to be executed for the energyconsuming unit comprises controlling the energy consuming unit accordingto the usage instructions.
 6. The system of claim 1, wherein the demandresponse strategy comprises scheduled usage for the energy consumingunit during each time window of day j+1.
 7. The system of claim 1,wherein the energy consuming unit is a HVAC system and the demandresponse strategy comprises thermostat settings for the HVAC system foreach time window of day j+1.
 8. A system for creating a demand responsestrategy for a building, the system comprising a computing devicecomprising: a memory configured for storing actual usage data associatedwith an energy consuming unit in the building for one or more timewindows on each of days i through j, wherein day i is the first day forwhich data is stored and j is the most recent day for which data isstored; and a processor configured for: receiving the actual usage datafrom the memory; executing at least one computer-based learning systemto model energy consumption for day j+1 based on at least the actualusage data for the energy consumption unit, and generating a demandresponse strategy for the energy consuming unit for day j+1 based on themodeled energy consumption and next-day energy pricing for each timewindow for day j+1.
 9. The system of claim 8, wherein: the energyconsuming unit is a regulatable energy consuming unit; and the processoris further configured for: receiving usage data associated withoperating a deferrable load energy consuming unit for each run cycle,receiving next-day energy pricing for each time window for day j+1, andgenerating a demand response strategy for the deferrable load energyconsuming unit, the demand response strategy for the deferrable loadenergy consuming unit comprising at least one time window in day j+1during which operation of the deferrable load energy consuming unit isallowed based at least on the usage data associated with the deferrableload energy consuming unit and the next-day energy pricing for day j+1.10. The system of claim 9, wherein the processor is further configuredfor electrically communicating the demand response strategy for day j+1associated with the regulatable energy consuming unit to a first controlreceiver and the demand response strategy for day j+1 associated withthe deferrable load energy consuming unit to a second control receiver,the first control receiver being associated with and in electricalcommunication with the regulatable energy consuming unit and the secondcontrol receiver being associated with and in electrical communicationwith the deferrable load energy consuming unit, the first and secondcontrol receivers configured for controlling operation of theregulatable energy consuming unit and the deferrable load energyconsuming unit, respectively, based on the communicated demand responsestrategies.
 11. The system of claim 9, wherein generating the demandresponse strategy for the deferrable load energy consuming unitcomprises executing a binary integer programming strategy.
 12. Thesystem of claim 8, wherein: the memory is configured for storing acurrent charge for a battery on day j, the battery configured forreceiving and storing energy generated by a photovoltaic system; and theprocessor is further configured for: receiving the current charge forthe battery from the memory, and generating a control and managementstrategy for the battery for day j+1 based at least on the currentcharge and next-day energy generation and consumption difference betweenat least one renewable energy source and at least one energy consumingunit, wherein the control strategy for the battery comprises identifyingone or more time windows during which the battery is charged when PVgeneration is higher than the demand or energy from the battery isdischarged for consumption by one or more energy consuming units when PVgeneration is below a predetermined level.
 13. The system of claim 8,wherein: the energy consumption unit is an HVAC system, the actual usagedata comprises an outdoor temperature for each time window on days Ithrough j, an indoor temperature for each time window on days i throughj, and a thermostat setting for each time window on days i through j,and the processor is further configured for using one or more of thecomputer based learning systems to model energy consumption of the HVACsystem for day j+1 based on the weather forecast for day j+1 and theactual usage data for days i through j.
 14. The system of claim 13,wherein generating the demand response strategy comprises generatingthermostat settings for one or more time windows on day j+1 such thatenergy costs for day j+1 are minimized.
 15. The system of claim 14,wherein the computer-based learning system comprises a neural networksystem.
 16. The system of claim 15, wherein the neural network systemcomprises a multilayer perceptron, and the multilayer perceptroncomprises: first, second, and third input nodes, the first input nodeconfigured for receiving from the memory the outdoor temperature foreach time window during each day i through j, the second input nodeconfigured for receiving from the memory the indoor temperature for eachtime window for each day i through j, and the third input nodeconfigured for receiving from the memory the thermostat setting for eachtime window for each day i through j; a computation layer comprising aplurality of computation nodes, the processor configured for propagatingsignals from the input nodes through the computation layer in a forwarddirection on a layer-by-layer basis; and an output layer comprising atleast one output node, the output node indicating energy consumption forthe HVAC system for each time window for day j+1.
 17. The system ofclaim 14, wherein the computer-based learning system comprises aregression based system.
 18. The system of claim 17, wherein theregression based system solves the following non-linear equation tolearn energy consumption of the HVAC system:Q _(k) =q(T _(k+1) ^(I) ,T _(k) ^(I) ,T _(k) ^(O),β) wherein Q_(k) isthe k^(th) HVAC energy consumption observation stored in the memory,q(•) is a third order polynomial function of a predictor variable vector(T_(k+1) ^(I),T_(k) ^(I),T_(k) ^(O)) which includes thermostat settings,indoor temperatures, and outdoor temperatures for observations k=1 . . .n, respectively, n is the number of observations saved in the memory,and β is a parameter vector that includes β₀, β₁, . . . β_(p).